Induction of a Novel Hybrid Decision Forest Model based on Information Theory

For the task of classification, the quality of rule set is usually evaluated as a whole rather than evaluating the quality of a single rule. The present investigation proposes a hybrid classifier named FDF. By redefining information gain from the general sense of Information theory, rule sets are built and combined to be decision forest by down-top learning strategy. The finial decision tree nodes contain univariate splits as regular decision trees, but the leaves contain Naive Bayes. Empirical studies on a set of natural domains show that FDF has clear advantages with respect to the probabilistic performance.